The present invention relates to a method and device for facial feature detection.
During the last decade, the field of face detection and facial feature detection has been constantly evolving. Face detection and facial feature detection have become popular areas of research in the field of computer vision and a basis for more complicated applications such as face recognition.
For example, in order to initiate the process of face recognition, usually the face first has to be detected. Clearly, an accurate detection of human faces in arbitrary scenes is a basic step in the process. Accurate detection of human faces for the input of digital images would enable many beneficial applications. Face recognition applications are just one example of applications that rely on face detection. However, other applications are also known. For example, in order to focus an image to be photographed, automatic and semi-automatic focusing cameras often focus on a portion of a scene. If the camera were able to locate faces in the scene, then the focus could be optimized for the faces, unless the photographer decides otherwise. An accurate detection of facial features may enable other beneficial applications and reduce the computational complexity of complex applications such as face recognition.
During the last few years, face detection has been studied in relation to the subject of image understanding. However, face detection remains an area with high computational requirements, particularly if an accurate recognition of the face outline and facial features is needed.
One example of a known face detection method is disclosed in U.S. Pat. No. 5,835,616, issued on Nov. 11, 1998. The patent discloses a two-step process for automatically identifying a human face in an electronically digitized image (for example, taken by a handheld, digital camera or a digital video-camera such as a camcorder), and for confirming the existence of the face in the image by examining facial features. Step 1 is, to detect the human face, is accomplished in stages that include enhancing the digital image with a blurring filter and edge enhancer in order to better distinguish the unique facial features such as wrinkles and curvature of the facial image. After prefiltering, preselected curves sometimes referred to as snakelets are superimposed on the image where they become aligned with the natural wrinkles and curves of the facial image. Step 2, to confirm the existence of a human face, is done in seven stages by finding facial features of the digital image encompassing the chin, sides of the face, virtual top of the face, eyes, mouth and nose. Ratios of the distances between these found facial features can be compared to previously stored reference ratios for facial recognition.
However, this method, as many other methods which are known in the face detection field, is computationally expensive and demands a substantial amount of memory allocation. In particular, since the size of the face (and the features therewithin) relative to the size of the digital image are not always known, the face detection process has to include repetitive search steps. The high number of iterations makes such a search computationally expensive. Moreover, the search has to include steps of matching the potential face or potential facial features with different translational, rotational, and scaling parameters. The matching requires the allocation of even more memory. In addition, such a technique requires a database of faces and/or facial features for the searching. Maintaining this database on a mobile telephone, for example, requires substantial amount of memory allocation.
One of the challenges in the field is to implement face detection methods in a computing unit with limited memory resources, and with limited computational abilities. A method that performs face detection and facial feature detection with low computational complexity is usually different from other face detection and recognition methods. In particular, the algorithms used, the limited memory and the character of the input digital image are different. In order to avoid high computational complexity when using computing units with limited memory and computational power, a few methods that do not use complex databases and algorithms have been developed.
One such method is disclosed in U.S. Pat. No. 6,697,502, issued on Feb. 24, 2004. The patent discloses a digital image processing method for detecting human figures in a digital color image. The first step of the disclosed method is providing a digital color image having pixels representing Red-Green-Blue (RGB) values. The second step is segmenting the image into non-overlapping regions of homogeneous color or texture. The following step is detecting candidate regions of human skin color and detecting candidate regions of human faces. Then, for each candidate face region, a human figure is constructed by grouping regions in the vicinity of the face region according to a predefined graphical model of a human figure, giving priority to human skin color regions.
U.S. Pat. No. 6,885,761 issued on Apr. 26, 2005 discloses a portable device which is adapted to identify human faces. The device includes an image input section which picks up a two-dimensional image containing a person's face, using an image sensor. A face area extracting section extracts the image of the face of the person from the image. A feature detection section detects the position of characteristic feature(s) of the face of that person. A face outline determining section determines a border between a face outline and a background. An image processing section generates a person's portrait in which the characteristic feature(s) of the face is emphasized.
Both U.S. Pat. No. 6,885,761 and U.S. Pat. No. 6,697,502 describe methods that can easily be used to detect human faces in a given digital image. However, the described methods cannot efficiency detect the facial features that comprise the detected face.
There is thus a widely recognized need for, and it would be highly advantageous to have, an apparatus and a method for face and facial feature detection for a computing unit with limited memory and computational power which are devoid of the above limitations.